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This  is  the  Final  Report  for  Contract  N00014-75-C-0838 , 
covering  a period  of  three  and  three-quarter  years  from  January  1975 
to  September  1978.  Research  was  performed  in  two  major  areas:  instruc- 
tional theory  and  development  of  a generalized  maintenance  trainer- 
simulator.  The  domain  of  instructional  theory  included  five  related 
research  projects.  These  were  the  effects  of  visual  analogies  of 
abstract  concepts,  Markov  decision  models  for  instructional  sequence 
optimization,  electrophysiological  correlates  of  cognitive  processing, 
cognitive  strategies  for  text  processing,  and  heuristic  techniques  for 
logical  problem  solution.  The  work  on  the  development  of  the  maintenance 
trainer- simulator  included  tests  of  the  system  in  Navy  School  environments 
and  led  to  a project  to  further  develop  the  system  for  regular  use  in 
Navy  schools. 


In  addition  to  producing  instructional  systems  which  are  now 
being  developed  for  use  in  Navy  schools,  this  contract  produced  fourteen 
technical  reports,  seven  chapters  in  books  or  articles  in  journals,  and 
nine  papers  for  professional  meetings. 
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RESEARCH  ON  LEARNING  STRATEGIES  AND 
HANDS-ON  TRAINING  IN  CA1 


I.  INTRODUCTION 

This  report  covers  a period  of  three  and  three-quart»r  years, 
from  1 January  1975  to  30  September  1978.  The  work  performed  during  this 
interval  was  concerned  with  two  major  research  areas:  i nstructional  theory 
and  development  of  a maintenance  trai ner-simulator . The  projects  related 
to  instructional  theory  were  visual  analogies  of  abstract  concepts, 

Markov  decision  models  for  instructional  sequence  optimization,  electro- 
physiological  correlates  of  cognitive  processing,  cognitive  strategies 
for  text  processing,  and  heuristic  techniques  for  logical  problem  solu- 
tion. The  research  on  these  instructional  theory  topics  was  primarily 
conducted  in  laboratory  environments.  The  research  on  maintenance 
trainer-simulators  included  testing  of  the  research  products  in  Navy 
school  environments. 

During  this  period,  14  technical  reports,  seven  chapters  in  books 
and  publications  in  professional  journals,  and  nine  papers  for  profes- 
sional meetings  were  produced.  The  maintenance  trainer-simulator  developed 
and  tested  under  this  contract  is  currently  being  adapted  for  use  in  Navy 
Class  C schools,  under  a contract  sponsored  by  the  Navy  Personnel  Research 
and  Development  Center  and  the  Defense  Advanced  Research  Projects  Agency. 
Further  applications  for  the  trainer  are  being  investigated. 

BTL  research  on  instructional  theory  during  the  almost  four-year 
period  of  the  contract  reflects  an  increasing  awareness  of  advances  in 
cognitive  processing  theory.  This  research  represents  an  attempt  to 
apply  these  advances  to  instructional  contexts.  Cognitive  models  and 


experimental  results  guided  the  evolution  of  our  research  in  this  area. 

With  the  exception  of  the  work  on  Markov  models  for  instructional  optimiza- 
tion, all  five  of  the  projects  carried  out  on  instructional  theory  were 
influenced  by  cognitive  psychology  models.  In  particular,  the  recent 
work  on  strategies  for  text  processing  was  guided  by  a cognitive  processing 
model,  developed  in  a schema  theory  framework,  of  the  reading  process. 

BTL  research  has  also  been  influenced  by  advances  in  instructional 
delivery  technology.  Moving  with  the  rapidly  evolving  computer  industry, 
more  powerful  yet  smaller  computer  systems  have  been  employed.  In  some 
cases,  BTL  staff  designed  and  built  hardware  configurations  for  computer- 
based  instruction  that  are  only  now  being  offered  off-the-shelf  by  com- 
puter companies  or  peripheral  manufacturers.  (For  example,  a computer 
interface  to  control  a microfiche  projector  was  developed  in  our  labo- 
ratory before  any  were  made  available  commercially.)  Although  seme  of 
the  research  done  under  this  contract  made  use  of  a large  time-share 
computer  system  specifically  designed  for  CAT  (the  PLATO  IV  system), 
there  has  been  a trend  for  our  research  to  move  away  from  the  use  of 
such  large  systems  toward  single-user  disk-based  microcomputer  systems. 
Future  BTL  efforts  are  likely  to  continue  to  exploit  the  low-cost,  reli- 
ability, and  availability  features  of  such  micro  systems.  In  addition, 
much  of  our  future  work  will  exploit  the  new  software  transportability 
features  of  the  UCSD  Pascal  operating  system. 

In  the  remaining  sections  of  this  report,  progress  in  each  of  six 
major  research  areas  is  reported.  The  first  five  topics  can  be  grouped 
under  the  general  heading  of  "Instructional  Theory  and  CAI."  The  last 
topic  deals  with  the  development  and  testing  of  the  Rigney  trainer- 
simulator. 
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II.  VISUAL  ANALOGIES  OF  ABSTRACT  CONCEPTS 

Three  technical  reports  and  a published  paper  in  the  Journal  of 
Educational  Psychology  (Rigney  & Lutz,  1976)  described  work  on  the  effects 
of  visual  analogies  from  1975  to  1978.  The  work  was  motivated,  in  part, 
by  a desire  to  explore  the  effects  of  right-hemisphere,  imaginal  pro- 
cessing on  retention  of  relatively  abstract  ideas.  Mental  imagery,  it 
was  felt,  could  be  expected  to  provide  a dual-coding  of  the  abstract 
concepts  to  be  learned,  when  conjoined  with  a traditional  verbal  exposi- 
tion of  the  concepts.  The  existence  of  such  a dual  code  in  the  mind  of 
a student  was  expected  to  improve  the  probability  of  recall  of  the  concep- 
tual material. 

The  experiments  to  test  these  hypotheses  made  use  of  special 
instructional  materials  developed  at  Behavioral  Technology  Laboratories, 
dealing  with  electrochemistry.  Imaginal  and  non-imaginal  technigues  were 
compared  in  two  contexts:  initial  acquisition  of  concepts  and  rehearsal 
of  concepts  for  recall.  Several  study  techniques  and  concept  presentation 
modes  were  studied,  including  experimenter-supplied  interactive  graphiis. 
additional  written  explanation  of  concepts,  student-generated  illustra- 
tions, student  paraphrases,  and  mentally  generated  verbal  analogies.  The 
results  of  the  series  of  studies  show  that,  in  the  case  of  initial  acqui- 
sition of  the  abstract  concept  used,  experimenter-supol ied  interactive 
graphics  resulted  in  significantly  better  recalls.  For  post-acquisition 
practice  with  the  concepts,  the  most  effective  means  of  improving  recalls 
was  a method  in  which  students  were  guided  to  reconstruct  a graphic 
illustration  of  the  concepts.  The  results  of  these  experiments  suggest 
guidelines  for  the  use  of  graphics  features  in  CAI. 
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TR  No.  75 


Joseph  W.  Rigney 
analogies 


S Kathy  A.  Lutz,  The  effects  of  interactive  graphic 
of  concepts  in  chemistry,  May  1975. 


Recent  advances  in  knowledge  about  the  different  information 
processing  functions  of  the  right  and  the  left  cerebral  hemispheres, 
and  laboratory  studies  of  these  different  functions  were  reviewed  for 
their  implications  for  training  and  education.  The  evidence  is  that 
the  right  hemisphere  is  specialized  for  spatial  and  topographic  imagery 
processes,  while  the  left  is  specialized  for  serial,  analytical,  lan- 
guage processes.  The  two  hemispheres  have  different  degrees  of  access 
to  the  vocalizing  system  on  the  midline,  which  is  primarily  under  the 
control  of  the  left  hemisphere,  and  to  the  motor  systems  for  the  right 
and  left  hands.  Under  certain  conditions,  the  left  hemisphere  can 
override  control  over  these  systems  by  the  right.  The  task  that  is 
given  to  the  subject,  the  orienting  task,  not  only  determines  the 
subsequent  information  processing  done  to  mediate  responding,  but  also 
influences  where  this  processing  is  done  in  the  cerebral  hemispheres. 

Recent  studies  of  the  effects  of  mental  imagery  on  learning  and  memory 
predominantly  find  strong  positive  effects.  In  verbal  learning  whatever 
mental  imagery  adds,  a spatial  organization,  an  integrating  context,  or 
a second  coding,  it  is  clear  that  it  results  in  better  retention  and 
recall  in  the  laboratory. 

The  implications  of  this  for  training  and  education  are  that 
the  right  hemisphere  functions  have  been  relatively  neglected  in  the 
predominantly  verbal  mediation  of  instruction  and  knowledge.  The 
challenge  to  instructional  technology  is  to  utilize  the  dual-coding 
system  more  effectively. 

The  first  of  several  planned  studies,  using  interactive,  animated 
graphics  for  illustrating  the  abstractions,  concepts  and  laws  of  science, 
and  for  stimulating  imaginal  processes  in  students,  is  described  in  this 
report.  It  was  found  that  these  graphics  did  have  positive  effects  on 
the  learning  of  concepts  in  electro-chemistry,  as  measured  by  recall 
tests  of  knowledge,  comorehension,  and  application,  and  that  the  lesson 
containing  these  graphics  was  more  attractive  to  students  than  a lesson 
in  which  purely  verbal  explanation  was  used.  Subjects  who  took  the 
lesson  providing  external  imagery  reported  experiencing  more  internal 
imagery  than  did  the  subjects  who  received  the  verbal  version. 

These  positive  outcomes  encourage  further  research  into  (1)  better 
methods  for  inducing  mental  imagery  in  students  than  verbal  instructions, 
(2)  theoretical  foundations  for  the  processing  of  external  imagery  relating 
it  to  current  theories  of  the  structure  of  long  term  memory,  and  (3) 
objective  indicators  of  mental  imaging  that  would  be  more  reliable  than 
verbal  reports  of  subjects. 


-4- 


T 


TR  No.  79 


Joseph 


...  Rigney  & Kathy  A.  Lutz,  The  effects  of  interactive  graphic 
analogies  of  concepts  in  science,  August  1976. 


In  this  series  of  three  studies,  interactive  granhics,  using 
the  plasma  panel,  touch  panel  interface  of  the  PLATO  system,  were  used 
to  simulate  the  topography  and  functions  of  a battery,  to  teach  ele- 
mentary concepts  in  electrochemistry.  The  effects  of  these  graphics 
were  compared  with  verbal  descriptions  of  the  topography  and  functions, 
in  otherwise  equivalent  lessons;  and  with  playing  checkers  for  the  same 
amount  of  time  as  used  by  postlesson  conditions  requiring  subjects  to 
reconstruct  the  graphic  simulation  from  memory,  or  to  attempt  to  imagine 
mental  images  of  these  graphics.  The  principal  conclusions  supported  by 
the  results  of  these  studies  are  as  follows: 

1.  As  used  here,  interactive  graphics  were  rated  by  students  to 
be  a more  interesting  way  to  present  information  about  abstract  concepts 
in  science,  and  therefore  may  have  received  more  attention  during  acqui- 
sition than  did  a purely  verbal  mode  of  presentation.  When  corrected 
for  the  effects  of  prior  knowledge,  mean  scores  on  verbal  and  -aphics 
posttests  were  slightly,  but  significantly  higher  for  lesson  and  oost- 
lesson  conditions  containing  interactive  graphics.  Mental  imagery  could 
be  another  source  of  these  effects.  Students  in  Experiment  I did  report 
that  the  graphics  in  the  lesson  induced  mental  imagery  while  they  were 
taking  the  lesson.  However,  the  attempts  of  students  in  Experiment  II 
to  recall  mental  images  of  lesson  material  did  not  increase  scores  on 
the  posttest,  in  comparison  to  the  control  group. 

2.  Interactive  graphics,  as  used  here,  evidently  are  most 
effective  auring  initial  acquisition.  Requiring  students  to  reconstruct 
the  graphic  simulation  after  the  initial  lesson  contributed  less  to 
verbal  posttest  performance.  However,  this  postlesson  condition  did 
result  in  the  acquisition  of  additional  information,  reflected  in  higher 
scores  on  a graphics  posttest  in  Experiment  III,  These  results  sugoest 
that  both  shallow  and  deep  processing,  in  the  Craik  and  Lockhart  sense, 
were  induced  by  this  postlesson  condition. 

3.  The  fact  that  students  were  able  to  deal  with  both  tooo- 
graphic  and  abstract  conceptual  information  in  either  verbal  or  graphic 
form,  suggests  either  the  same  deep  structures  in  LTS,  or  rapid  trans- 
formation of  different  structures  in  LTS  into  a common  representation 
when  necessary.  The  current  version  of  semantic  network  theory  of  Long 
Term  Memory  (Rumelhart  and  Ortony,  1976),  gives  a plausible  account  of 
this  capability  in  terms  of  a common  basis  for  storage  and  processing 
of  both  graphic  and  verbal  information. 

4.  It  should  be  pointed  out  that  these  studies  dealt  with  fixed 
effects  variables,  and  that  there  are  many  other  relationships  between 
graphics  and  verbal  modes  of  presentation  that  were  not  explored  here. 
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In  some  of  these,  one  mode  would  complement  the  other,  so  that  both 
would  be  required  to  convey  the  essential  information  or  to  enable  the 
desired  performance.  Such  would  be  the  case  where  interactive  graphics 
are  used  to  represent  the  front  panels  of  equipment  and  students  operate 
on  these  interactive  graphics  in  ways  analogous  to  operating  on  the 
actual  equipment  front  panels.  Or,  the  graphics  might  contain  informa- 
tion only  referred  to  verbally,  a common  usage  for  diagrams  or  photo- 
graphs in  texts. 

In  other  cases,  the  dynamics  of  processes  might  be  so  complex 
that  verbal  description  becomes  too  long  and  clumsy.  Animated  graphics 
would  allow  the  student  to  attend  to  different  parts  of  the  animation 
at  will,  and  rapidly  comprehend  the  implications  of  the  analogy  or 
metaphor.  The  internal  workings  of  a transistor  come  to  mind  as  an 
example.  Here,  it  might  be  advantageous  to  depict  electrons  and  holes 
"flowing,"  junction  phenomena  changing,  and  the  "flow"  of  currents  in 
forward  and  reverse  biased  loops  all  at  once,  so  that  the  student  could 
integrate  these  events  rather  quickly  into  an  overall  comprehension  of 
how  a transistor  works.  In  this  case,  simultaneous  animation  of  many 
related  events  would  be  a unique  contribution  to  dynamic  graphics,  since 
verbal  description  is  necessarily  serial.  This  usage  of  animated  graphics 
would,  however,  require  a plasma  panel  capable  of  writing  and  erasing  at 
higher  data  rates  than  currently  available. 


TR  No.  82 

Kathy  A.  Lutz  & Joseph  W.  Rigney,  The  effects  of  student-aenerated 
elaboration  during  acquisition  of  concepts  in  science, 

September  1977. 

This  is  the  last  of  a series  of  studies  of  experimenter  and  student 
supplied  cognitive  strategies  for  acquisition  and  retention  of  concents 
in  science.  A lesson  in  electrochemistry  was  used  as  the  subject  matter 
in  these  studies.  The  conclusions  we  believe  are  warranted  by  the  re- 
sults of  these  studies  as  follows.  The  most  effective  strategy  for  facil- 
itating acquisition  consisted  of  experimenter-suppl ied,  interactive  graphic 
analogies  that  supplemented  verbal  description,  in  comparison  to  additional 
verbal  explanation.  Two  explanations  for  these  positive  effects  would  be 
that  these  graphic  analogies  captured  the  students'  attention  more  effec- 
tively, and  that  they  provided  an  overall  structure,  by  which  individual 
concepts  could  be  interrelated,  serving  as  a type  of  organizer. 

Student-supplied  elaboration,  in  the  form  of  paraphrasing,  drawing 
i 1 1 ustrations ,or  thinking  of  verbal  analogies  for  verbally  described 
concepts  in  the  lesson,  was  less  effective  during  initial  acquisition 
than  the  experimenter-suppl ied  graphic  analogies,  and  no  more  effective 
than  using  the  same  amount  of  time  to  go  through  the  lesson  twice.  How- 
ever, drawing  illustrations  of  concepts  in  the  lesson  was  the  most  effec- 
tive of  these  elaboration  strategies  for  students  with  little  or  no  prior 
knowledge  of  the  subject  matter. 
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After  acquisition  coding  has  occurred,  the  most  effective  strategy 
investigated  in  these  studies  was  a postlesson  review  in  which  students 
were  required  to  reconstruct  the  complete  graphic  illustration  of  the 
electrochemistry  of  a battery,  by  touching  appropriate  spots  on  the  dis- 
play screen.  This  review  may  have  forced  the  students  to  do  orocessina 
that  integrated  the  various  electrochemical  concepts.  Attempts  to  induce 
students  to  review  the  lesson  by  generating  mental  imagery  in  which  they 
visualized  the  flow  of  events  in  a battery  was  not  an  effective  review 
strategy. 
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III.  MARKOV  DECISION  MODELS  FOR  INSTRUCTIONAL 
SEQUENCE  OPTIMIZATION 


Two  technical  reports  extended  an  earlier  model  (Wollmer,  R.  D. 

A Markov  Decision  Model  for  Computer-Aided  Instruction.  Los  Angeles: 
University  of  Southern  California,  Behavioral  Technology  Laboratories, 
December,  1973,  Technical  Report  No.  72),  produced  under  ONR  Contract 
N00014-67-A-0269-0025.  This  research  was  conducted  from  the  onset  of  the 
contract  until  March  1976.  The  first  of  the  two  reports  w.s  designed  to 
evaluate  the  Wollmer  model  in  a CAI  context.  Two  experiments  were  per- 
formed to  test  the  effect  on  learning  of  using  the  model  to  optimize 
practice  schedules.  The  results  did  not  reveal  a significant  improvement 
in  amount  learned  when  the  optimization  system  was  used.  These  results 
suggest,  first,  that  number  of  practice  trials  may  be  less  important  than 
other  factors  for  complex  tasks,  and,  second,  that  complex  tasks  may 
require  very  large  numbers  of  practice  trials  to  ensure  significant  prac- 
tice effects. 

The  second  technical  report,  which  was  the  last  in  the  series, 
further  developed  and  refined  the  Wollmer  approach.  A mathematical  treat- 
ment of  partially  observable  Markov  desision  processes  using  an  infinite 
planning  horizon  was  developed.  Although  the  system  was  developed  for 
computer-aided  instruction  applications,  it  has  wider  applicability. 
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TR  No.  76 


Richard  D.  Wollmer  & Nicholas  A.  Bond,  Evaluation  of  a Markov-decision 

model  for  instructional  sequence  optimization,  October  1975. 

Wollmer' s Markov  Decision  Model  for  instructional  sequence 
optimization  was  investigated  in  a computer-assisted  instruction  (CA1) 
context.  Two  special  CAI  programs  served  as  vehicles  for  testing  the 
model;  one  program  (K-laws)  taught  the  students  to  solve  DC  circuit 
problems  using  Kirchhoff's  Laws;  the  other  (TRIG)  gave  practice  in 
manipulating  the  six  tri gonometric  ratios.  The  K-Laws  course  had 
elevan  stages  or  levels;  TRIG  had  five;  both  courses  were  arranged  in 
a hierarchical  order.  The  Wollmer  model  requires  that  transfer  would 
occur  from  one  stage  to  the  next  in  a hiearachical  learning  sequence, 
and  that  these  effects  could  be  estimated  so  as  to  produce  an  optimal 
training  schedule.  To  determine  the  effects  of  additional  practice, 
half  the  calibration  sample  was  required  to  finish  one  successful  trial 
and  half  were  required  to  have  two  successes,  before  moving  on  to  the 
next  stage.  Thirty  subjects  took  the  K-Laws  course,  80  completed  TRIG 
Instruction  was  given  at  individual  CAI  terminals. 

All  subjects  finished  the  course,  and  learned  to  perform  satisfac- 
torily the  final  criterion  behaviors.  Practice  effects  were  unexpectedly 
slight;  people  who  had  one  success  at  each  stage  of  the  course  had  about 
the  sane  cri terion-problem  performance  as  those  who  had  two  successes 
throughout.  The  average  time  required  to  achieve  a second  success  was 
not  appreciably  different  from  that  required  for  the  first,  and  two 
successes  at  the  immediately  preceding  level  was  no  better  than  one.  as 
far  as  transfer  to  the  next  higher  staae  was  concerned.  These  results 
indicated  that  the  Wollmer  hierarchical  model  could  not  improve  overall 
learning  much  by  "optimal"  scheduling  of  practice. 

One  implication  of  the  findings  is  that  in  complex  learning  hierar- 
chies where  the  top  or  most  difficult  task  consists  of  a collection  of 
previously- learned  skills,  performance  time  on  that  top  task  may  be  more 
dependent  on  the  number  of  subski  11s  involved  than  upon  the  number  of 
practice  trials  in  preceding  stages.  Another  implication  is  that  if 
practice  and  transfer  effects  are  to  be  significant  in  learning  this 
kind  of  hierarchical ly-structured  material,  then  a very  larqe  number  of 
practice  trials  may  be  necessary. 
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TR  No.  77 


Richard  D.  Wollmer,  Partially  observable  Markov  decision  processes  ovc-m 
an  infinite  planning  horizon  with  discounting,  March  1976. 


This  is  the  last  in  a series  of  technical  reports  concerned  with 
mathematical  approaches  to  instructional  sequence  optimization  in 
instructional  systems.  The  problem  treated  here  is  very  closely  re- 
lated to  that  treated  by  Smallwood  and  Sondik  (4).  Both  papers  deal 
with  Markov  decision  processes  where  the  true  state  of  the  system  is 
not  known  with  certainty.  Hence  the  state  of  the  system  is  characterized 
by  a probability  vector.  Each  action  yields  an  expected  reward,  trans- 
forms the  system  to  a new  state  and  yields  an  observable  outcome.  One 
wishes  to  determine  an  action  for  each  probability  state  vector  so  as 
to  maximize  the  total  expected  reward.  Smallwood  and  Sondik  (4)  solve 
this  proble-  exactly  for  a finite  time  horizon.  This  report  treats 
the  infinite  time  horizon  with  a discount  factor,  using  a partial  N 
dimensional  Maclaurin  series  to  approximate  the  total  ODtimal  reward 
as  a function  of  the  probability  state  vector.  While  this  model  was 
developed  for  computer  aided  instruction,  it  is  applicable  to  other 
situations  as  well.  This  model  also  is  of  considerable  theoretical 
val ue. 


% 

-10- 


i 


IV.  ELECTROPHYS IOLOGICAL  CORRELATES  OF 

COGNITIVE  PROCESSING  , 

This  line  of  research,  which  had  begun  before  the  inception  of 
the  contract  here  reported,  culminated  in  July,  1973,  in  a technical 
report  describing  the  results  of  an  arduous  series  of  experiments. 

Special  purpose  data  collection  instruments  were  devised  and  built, 
electrical  noise  shielding  was  designed  and  implemented,  and  special 
software  for  data  collection  and  analysis  was  written  and  tested. 

The  goal  of  the  research  was  to  relate  electrophysiological 
phenomena  to  ongoing  cognitive  activities.  Subjects  were  required  to 
solve  five-letter  anagrams.  Solution  achievement  was  signalled  by 
pressing  a microswitch.  Electroencephalographic  activity  and  time  to 
solution  were  recorded  from  the  onset  of  each  stimulus.  Sixty  stimulus 
anagrams  were  selected  from  lists  of  abstractness  and  usage  frequency 
to  form  four  stimulus  groups:  1)  concrete/high  frequency,  2)  concrete/ 
low  frequency,  3)  abstract/high  frequency,  and  4)  abstract/low  frequency. 

Analysis  of  the  solution  time  data  showed  that  abstractness  and  frequency 
both  had  significant  effects,  with  no  interaction.  Analysis  of  the 

* | 

electroencephalographic  data  showed  that  anagram  solution  and  recogni- 
tion of  non-scrambled  words  were  both  followed  by  a positive  shift,  which 
may  be  related  to  decision  processes. 


TR  No.  86 


Louis  A.  Williams  & Joseph  W.  Rigney,  Electrophysioloqical  correlates  of 
cognitive  activity:  Event  related  slow-potentials  developed  during 
solution  of  anagrams,  July  1978. 


Electroencephalographic  (EEG)  activity  and  time  to  solution  we>-e 
recorded  for  sixteen  human  subjects  during  a task  requiring  the  solu- 
tion of  five  letter  anagrams.  Solution  achievement  was  signaled  by 
depressing  a microswitch.  The  sixty  stimulus  anagrams  were  selected 
from  lists  of  abstractness  and  usage  frequency  to  form  four  stimulus 
groups:  1)  concrete/high  frequency,  2)  concrete/low  frequency, 

3)  abstract/high  frequency,  4)  abstract/low  frequency.  Stimulus  words 
were  presented  to  subjects  in  a randomized  order  interspersed  with  a 
non-anagram,  recognition  word  (TANGO)  or  a blank  screen  (BLANK).  Stimulus 
presentation  was  under  computer  control  and  displayed  upon  a compute-'  C!  T 
Solution  time  was  subjected  to  an  analysis  of  variance.  Abstractness  an 
frequency  were  both  significant.  Abstractness  had  the  greater  effect 
upon  solution  time.  There  were  no  interaction  effects.  Concrete  (low 
abstractness)  anagrams  were  solved  more  quickly  than  abstract  anagrams, 
with  the  effect  of  frequency  of  usage  additive  to  solution  times.  This 
result  was  concluded  to  support,  but  not  confirm  a parallel  processing 
hypothesis:  conscious  processing  concerned  with  anagram  letter  re- 
arrangement, and  simultaneous  unconscious  processing  concerned  with 
retrieval  of  possible  solution  words  from  long  term  memory. 

The  EEG  was  analyzed  by  Fourier  methods  to  determine  frequency 
and  amplitude  content.  A coherence  analysis  was  performed  upon 
selected  segments  of  the  EEG  pre  and  post  response.  Visual  analysis 
of  individual  trials  was  accomplished  through  a computer  developed 
super-imposition  display.  Displayed  trials  were  organized  by  correct 
or  incorrect  solutions,  failure  to  achieve  solution  (TIME-OUT),  TANGO 
or  BLANK  presentations.  The  development  of  a negative  shift  following 
stimulus  onset  in  all  except  BLANK  trials  wa s revealed.  Trials  in  which 
a correct  solution  was  achieved,  or  TANGO  recognition  occurred,  showed 
a reactive  shift  to  positivity  at  about  350  milliseconds  latency.  This 
identification  was  supported  by  the  analysis  of  coherence.  The  negative 
shift  was  concluded  to  be  similar  to  the  Contingent  Negative  Variation 
(CNV).  The  positive  shift  was  concluded  to  be  a P300  wave.  The  CNV- 
like  shift  was  related  to  selective  attention-selective  responding 
demands  of  the  paradigm.  The  P300  was  related  to  decision  processes 
allowing  a relaxation  of  attention  and  responding. 


-12- 


V.  COGNITIVE  STRATEGIES  FOR  TEXT  PRESSING 


Work  in  this  area  was  carried  out  from  January  of  1976  to  the 
end  of  the  contract.  Four  technical  reports  and  four  chapters  in  books 
describe  the  results  of  the  research.  (The  last  of  these  reports,  Tech. 
Rep.  No.  85,  was  co-sponsored  by  the  Defense  Advanced  Research  Projects 
Agency  under  contract  N00014-77-C-0328. ) These  technical  reoorts  mark 
four  stages  in  the  inception  of  a large  effort  to  understand  the  nature 
of  some  of  the  higher  cognitive  processes  involved  in  reading  and  to 
apply  the  findings  to  the  development  of  training  programs  for  job- 
related  reading. 

In  the  first  of  the  four  reports  (Tech.  Rep.  No.  78),  an  extensive 
survey  was  made  of  many  recent  trends  in  cognitive  science  and  neuro- 
science which  could  be  expected  to  have  an  impact  on  instructional  theory 
and  practice.  One  of  the  many  concepts  introduced  in  this  Daper  was  that 
of  the  relevance  of  text-type  to  reading  in  general  and  job-related 
reading  in  particular.  It  was  suggested  that  not  all  texts  are  of  the 
same  type--they  have  different  structural  and  semantic  characteristics 
which  can  be  expected  to  have  implications  for  what  and  how  much  of  the 
information  they  contain  will  be  remembered  after  reading.  The  second 
report  in  this  series  (Tech.  Rep.  No.  80)  expanded  upon  this  theme, 
presenting  a theoretical  viewpoint  on  the  role  of  text-type  perceptions 
in  reading.  A number  of  reading  strategies  were  discussed  within  the 
theoretical  context  propounded.  In  the  third  paper  (Tech.  Rep.  No.  81), 
the  theoretical  underpinnings  of  the  preceding  paper  were  examined  in 
greater  detail.  The  theoretical  framework  was  extended  to  cognitive 
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tasks  other  than  reading,  as  well.  In  the  fourth  report  of  the  series 

(Tech.  Rep.  No.  85),  the  theoretical  mechanisms  developed  in  previous  • 

reports  are  applied  to  a particular  experimental  situation  and  predictions 

for  the  memorability  of  propositions  in  different  types  of  texts  are  made. 

The  results  of  a series  of  experiments  designed  to  test  these  predictions 
are  reported,  the  analysis  of  which  supports  the  major  aspects  of  the 
theory.  The  experimental  results  suggest  that  the  theoretical  approach 
may  serve  to  generate  useful  reading  strategies,  sensitive  to  the  type 
of  text  being  read,  which  can  improve  learning  fro-  texts  of  different 
types  significantly. 


TR  No.  78 

Joseph  W.  Rigney,  On  cognitive  strategies  for  facilitating  acquisition, 
retention,  and  retrieval  in  training  and  education.  May  1976. 


The  idea  that  students  could  be  taught  to  be  more  effective  learners, 
in  distinction  to  being  taught  subject  matter,  is  explored  in  relation  to 
recent  advances  in  the  cognitive  and  neurosciences,  with  the  objective  of 
integrating  information  from  these  sources  into  a unified  viewDOint  that 
could  serve  as  a roadmap  for  research  and  as  a context  for  discussion. 

According  to  this  viewpoint,  cognitive  strategies  for  facilitating 
acquisition,  retention,  and  retrieval  of  information  and  of  performance 
are  composed  of  specifications,  called  orienting  tasks,  for  how  cognitive 
processes  are  to  be  used,  and  of  cognitive  processes  drawn  from  repre- 
sentational, selectional,  and  self-directional  resources.  Representa- 
tional resources  include  proposi tional  and  apposi tional  processes  of  the 
left  and  right  cerebral  hemispheres,  chiefly  language  and  imagery. 
Selectional  resources  consist  of  attentional  and  intentional  processes. 
Self-directional  resources  include  self-programming  and  self-monitoring 
processes.  Possibilities  for  teaching  students  better  control  over 
attentional  and  intentional  processes, by  using  neurophysiological 
indicators,  particularly  to  reduce  self-generated  distractions  during 
learning,  are  noted. 
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Concepts  of  processing  capacity,  depth  of  processing,  expended  Dro- 
cessing  capacity,  resource-limited  and  data-limited  processing,  top 
down-bottom  up  processing,  and  graceful  degradation  of  output,  advanced 
by  several  different  theorists,  are  considered  in  terms  of  possible  sources 
of  individual  differences,  possible  electrophysiological  indicators,  and 
implications  for  cognitive  strategies.  The  importance  of  long  term 
memory  in  learning  is  recognized.  Three  types  of  long  term  store  (LTS); 
semantic,  episodic  and  motor,  are  described.  Semantic  and  episodic  LTS 
are  of  great  current  theoretical  interest.  Their  imp! ications , particu- 
larly of  semantic  LTS,  for  acquisition,  retention,  and  retrieval  are 
discussed.  Norman's  concept  of  web-learning  is  an  example.  The  additional 
requirement  for  some  kind  of  LTS  for  controlling  skilled  performances  is 
noted,  and  two  recent  theoretical  formulations  of  cerebral -cerebell ar 
roles  in  this  regard  are  reviewed. 

Several  kinds  of  subject-matter  are  described  under  information  and 
performance  and  these  are  tentatively  cross-classified  with  types  of 
long  term  memory  and  strategies  for  acquisition,  retention,  and  retrieval. 
Different  approaches  to  teaching  students  cognitive  strategies  are 
described  in  terms  of  possible  combinations  of  instructional  control  and 
explicitness  of  the  strategy.  Finally,  techniques  for  implementation  of 
aopropriate  approaches  are  considered.  The  Learning  Assistance  Center 
(LAC)  concept  is  viewed  as  the  context  for  applying  implementation 
techniques  in  a systematic  and  long-term  fashion;  including  diagnosis 
of  individual  student  requirements  and  resources,  prescription  of  an 
aopropriate  combination  of  strategies,  and  instruction  in  how  to  use 
them.  It  is  proposed  that  LAC 1 s might  eventually  be  recognized  as  impor- 
tant resources  in  Naval  training. 


TR  No.  80 

Joseph  W.  Rigney  & Allen  Munro,  On  cognitive  strategies  for  processing 
text,  March  1977. 


Current  research  on  reading  is  hampered  by  the  lack  of  a frame- 
work within  which  to  study  the  effects  of  a reader's  prior  knowledge 
on  his  or  her  processing  of  an  unfamiliar  text.  As  a result,  most 
reading  research  has  emphasized  perceptual  rather  than  conceptual 
processing  during  reading.  Evidence  is  cited  in  support  of  the  claim 
that  various  types  of  prior  knowledge  play  important  roles  in  under- 
standing during  text  processing. 

Recent  developments  in  cognitive  psychology  and  artificial 
intelligence  have  resulted  in  a new  kind  of  model  for  conceptual  pro- 
cessing, called  procedural  semantics.  In  this  report,  a framework  is 
laid  for  the  application  of  the  procedural  semantics  formalism  to  the 
analysis  of  conceptually-driven  processing  in  reading.  According  to 
this  theory,  two  different  types  of  conceptual  processing  units 
(called  schemata ) are  responsible  for  conceptually-driven  processing 
in  reading.  One  type  is  the  form-schema,  which  accounts  for  the 


syntactic  or  formal  expectations  which  people  make  use  of  in  text 
processing.  The  other  type  is  the  content-schema,  which  accounts  for 
the  nature  of  readers'  semantic  expectations.  Models  for  a small  number 
of  specific  form-  and  content-schemata  are  proposed,  and  certain  exDeri- 
mental  and  observational  evidence  is  explained  in  terms  of  these  models. 

Implications  for  effective  reading  strategies  for  adult  reading  are 
derived  from  the  premises  of  the  model.  Several  different  kinds  of 
reading  strategies  are  characterized  in  terms  of  the  model.  When 
readers  employ  single-pass  strategies,  they  process  the  text  in  a 
strictly  linear,  left-to-right  fashion.  This  approach  makes  minimal 
use  of  the  potential  for  conceptually-driven  processing  that  could  be 
achieved  through  the  activation  of  some  high-level  schemata.  In 
exhausti ve  multi-pass  processing,  the  first  pass  results  in  the  activa- 
tion of  a number  of  form-  and  content-schemata  which  can  serve  as  an 
aid  in  subsequent  passes,  deriving  expectations  about  the  form  and 
meaning  of  what  is  about  to  be  read.  This  technique  can  ofter  be 
wasteful  of  resources,  since  it  does  not  actively  direct  processing 
toward  what  is  most  important  or  least  well  understood.  Extractive 
multi-pass  processing  reflects  a more  efficient  strategy  for  reading 
an  entire  text.  By  using  this  technique,  a reader  "skims"  the  text  in 
a selective  way  on  repeated  passes,  building  up  such  a complete  under- 
standing of  the  meaning  of  the  text  that  the  final  reading  of  the  text 
is  often  a process  of  merely  filling  in  the  gaps  in  understanding. 

This  technique  is  often  effectivelv  used  by  graduates  of  adult  reading 
improvement  classes.  Selective,  multi-pass  strategies  characterize  the 
reading  of  those  who  know  what  it  is  they  want  to  know,  and  who  are 
under  no  constraint  to  learn  all  that  might  be  learned  from  a text. 

In  this  type  of  text  processing,  the  reader  begins  the  task  with  the 
intention  of  acquiring  some  specific  information.  As  a result,  a 
number  of  specific  content-schemata  are  activated  and  are  used  to  guide 
the  order  and  the  selection  of  those  portions  of  the  text  to  be  pro- 
cessed. 

Several  potential  applications  are  suggested  by  the  consequences 
of  the  theory  for  conceptually-driven  processing  in  reading  presented 
here.  These  include  possible  uses  for  headings  in  texts,  means  for 
constructing  advance  organizers  for  texts,  and  traininq  readers  to 
make  more  effective  use  of  texts  by  being  sensitive  to  their  motivating 
tasks  and  by  exploiting  their  capacities  for  generating  expectations 
about  the  meaning  of  the  texts  through  conceptually-driven  Drocessing. 


TR  No.  81 

Allen  Munro  & Joseph  W.  Rigney,  A schema  theory  account  of  some  cognitive 
processes  in  complex  learning,  July  1977. 

Procedural  semantics  models  have  diminished  the  distinction  between 
data  structures  and  procedures  in  computer  simulations  of  human  intel- 
ligence. This  development  has  theoretical  consequences  for  models  of 
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cognition.  One  type  of  procedural  semantics  model,  called  schema  theory, 
is  presented,  and  a variety  of  cognitive  processes  are  explained  in 
terms  of  the  theory.  In  schema  theory,  the  flow  of  processing  control 
is  determined  not  by  a central  monitor,  but  by  interactions  among  the 
conceptual  entities  (schemata)  that  make  up  the  model.  Schemata  interact 
by  providing  activation  resources  to  each  other. 

Instantiation  is  the  special  process  whereby  a partial  copy  of  a 
strongly  activated  schema  is  created.  In  this  copy,  the  variables  of 
the  schema  are  filled  with  particular  values.  Such  copies  make  up 
specific  or  episodic  memory.  The  schemata  on  which  they  are  based 
comprise  generic  or  semantic  memory. 

Many  of  the  phenomena  of  consciousness  and  of  short-term  and  long- 
term memory  are  explained  on  the  basis  of  the  activation  processes  of 
schema  theory.  Unactivated  schemata  are  equivalent  to  all  the  unconscious 
knowledge  in  a person's  long-term  memory.  Schemata  that  are  activated, 
but  are  below  the  threshold  of  instantiation,  are  in  a preconscious 
or  subconscious  state.  Those  schemata  that  are  more  fully  activated, 
that  are  above  the  instantiation  threshold,  are  the  stuff  of  conscious 
thought,  and  may  be  thought  of  as  roughly  equivalent  to  the  contents  of 
short-term  memory. 

Conscious  cognitive  strategies  are  treated  as  the  activations  of 
abstract  prescriptive  schemata.  A treatment  of  creativity  is  presented, 
along  with  the  outlines  of  an  approach  to  individual  differences  in 
creativity.  The  effects  of  orienting  tasks  are  explained  in  schema 
theory,  and  the  relationship  between  orienting  tasks  and  self-direction 
in  complex  learning  and  problem  solving  is  discussed. 

Inference  and  depth  of  processing  receive  related  schema  theory 
treatments.  Both  concepts  are  treated  in  terms  of  the  extent  to  which 
activation  spreads  to  include  related  schemata.  In  general,  the  more 
schemata  activated  to  the  level  of  instantiation  by  some  datum,  the 
more  deeply  processed  that  datum  is.  Inference  is  seen  as  a kind  of 
delayed  deeper  processing. 

Types  of  insight  phenomena  from  several  contexts  can  also  be 
treated  in  schema  theory.  Each  type  of  insight  involves  the  instantia- 
tion of  one  or  more  new  schemata  that  take  some  pre-existing  concept 
in  memory  as  a parameter. 

Three  dimensions  for  distinguishing  or  comparing  schemata  are 
proposed:  function,  abstractness,  and  scope.  The  contrasts  between 
multi-store  models  of  cognition  and  schema  theory  are  summarized. 
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TR  No.  85 


Lynn  Gordon,  Allen  Munro,  Joseph  W.  Rigney,  8,  Kathy  A.  Lutz, 

Summaries  and  recalls  for  three  types  of  texts.  May  1978. 

A theoretical  orientation  for  the  study  of  different  types  of 
texts  is  presented.  Schema  theory  is  proposed  as  a useful  meta-theory 
within  which  to  develop  specific  theories  about  reading.  Both  theories 
about  processes  of  reading  and  theories  about  the  structure  of  what  is 
read  can  be  readily  formulated  in  schema  theory  terms.  It  is  proposed 
that  readers  make  judgments  about  the  types  of  texts  that  they  read  and 
that  these  judgments  bring  about  the  activation  of  expectations  wit1*! 
respect  to  the  structure  and  meaning  of  these  texts. 

Previous  work  on  the  structure  of  texts,  primarily  for  simple 
narratives,  is  reviewed.  Problems  with  earlier  formalisms  and  scoring 
methods  are  discussed,  and  heuristics  for  avoiding  these  problems  are 
presented . 

Three  types  of  texts  were  selected  for  study.  One  type  was  the 
simple  short  story,  a type  closely  related  to  (and,  in  some  cases, 
identical  with)  the  kinds  of  texts  studied  by  other  researchers.  The 
second  type  studied  were  instructions.  The  third  type  was  definitional 
explanations,  a type  well  characterized  by  popular  science  articles. 
Detailed  analyses  of  the  text  structures  and  text  semantics  for  eight 
texts  (three  stories,  two  instructions,  and  three  definitions)  are 
presented.  Texts  of  the  different  types  differ  from  each  other  in 
consistent  ways  on  two  dimensions.  First,  the  text  structures  of 
definitions  tend  to  be  organized  horizontally  rather  than  vertically, 
as  are  the  text  structures  of  stories  and  instructions.  Second,  the 
semantic  representations  of  stories  are  composed  of  specific  concepts, 
in  schema  theory  terms,  while  the  semantic  representations  of  instruc- 
tions and  definitions  consist  primarily  of  generic  concepts.  Or.  he 
basis  of  these  differences  among  the  texts,  we  predicted  that  stories 
would  be  better  remembered  than  instructions,  which  would,  in  turn, 
be  better  remembered  than  definitions.  Three  experiments  were  con- 
ducted to  test  this  hypothesis. 

In  Experiment  1,  subjects  read  and  summarized  six  texts  and 
later  recalled  three  of  these  texts.  Analysis  of  the  summary  data 
indicates  that  texts  of  different  types  are  summarized  to  about  the 
same  extent.  The  recall  data,  however,  suggests  that  text  type  may 
determine  the  amount  recalled.  Analysis  of  the  recall  data  showed  that, 
although  stories  were  remembered  best  (as  had  been  predicted),  the 
proposi tional  content  of  definitions  was  remembered  better  than  that  of 
instructions.  It  was  hypothesized  that  rereading  and  summarizing  may 
have  had  a differentially  facilitative  effect  for  later  recall,  bene- 
fiting the  recall  of  definitions  more  than  instructions. 
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In  order  to  test  this  hypothesis,  Experiments  2 and  3 were 
performed.  Subjects  heard  tape  recorded  texts  (in  Experiment  2 the 
same  set  of  texts  used  in  Experiment  1;  in  Experiment  3 a somewhat 
different  set),  and,  after  performing  a brief  interfering  task, 
recalled  each  text  after  hearing  it.  They  were  therefore  not  able  to 
reprocess  texts  as  they  had  been  able  to  in  Experiment  1.  In  general, 
the  results  of  these  experiments  confirmed  our  predictions:  stories 
were  recalled  better  than  instructions,  which,  in  turn,  were  recalled 
better  than  definitions.  Subjects'  recalls  in  these  experiments  were 
also  scored  for  the  amount  of  reorderi ng  of  the  textual  material. 

This  analysis  showed  a very  powerful  effect  due  to  text  type.  Recalls 
of  definitions  showed  significantly  more  reordering  than  did  recalls 
of  instructions,  which,  in  turn,  had  more  reordering  than  did  the 
recalls  of  stories.  These  results  are  also  in  accord  with  our  theory 
that  stories  have  more  hierarchi cal , differentiated  text  structures 
than  do  instructions  or  definitions,  and  that  definitions  have  less 
hierarchical  structures  than  do  instructions. 

Subjects  in  these  two  experiments  were  also  requested  to  cluster 
the  texts  in  natural  groups  according  to  their  types,  as  they  per- 
ceived them.  Their  groupings  were  remarkably  consistent  with  our  own 
cl assi f i cations . 

The  research  presented  demonstrates  the  need  for  a more  thorough 
investigation  both  of  the  nature  of  people's  expectations  for  differences 
in  different  types  of  text,  and  of  the  effects  of  such  expectations 
on  understanding  and  memory.  Further  research  is  also  needed  to  explore 
the  hypothesis  that  texts  of  different  types  may  benefit  differentially 
from  the  application  of  particular  learning  strategies,  such  as  re- 
reading and  summarizing. 
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VI.  HEURISTIC  TECHNIQUES  FOR  LOGICAL 
PROBLEM  SOLUTION 

This  research  effort  was  pursued  from  February  of  1977  to  June 
1978.  Ongoing  BTL  interest  in  the  nature  of  cognitive  processing 
during  troubleshooting  activities  motivated  a series  of  studies  on 
how  people  solve  verbal  logic  problems.  Two  technical  reports  were 
produced.  The  first  report  describes  an  initial  evaluation  of  two 
computer  programs  designed  to  aid  human  problem  solvers.  Experience 
with  these  programs  and  with  the  introspective  reports  of  logic  proble~ 
solvers  led  to  the  posing  of  two  basic  questions:  first,  what  factors 
in  these  problems  are  predictive  of  their  difficulty,  and,  second, 
what  strategies  can  be  taught  to  improve  problem  solving  performance? 

The  experiments  reported  in  the  second  technical  report  (No.  87) 
provided  partial  answers  to  these  questions.  It  appears  that  one  of 
the  most  important  predictors  of  success  in  the  solution  of  these 
problems  was  the  subject's  ability  to  convert  the  verbal  reoresentation 
to  some  more  logically  manipulate  representation.  Those  problems 
which  were  couched  in  such  a way  that  such  a translation  was  made  more 
difficult  were  harder  to  solve.  A training  system  for  improving  logic 
problem  solution  was  devised  and  tested.  Experimental  results  show 
that  use  of  the  techniques  did  improve  performance.  Future  efforts  in 
this  domain  should  concentrate  on  transporting  the  training  system 
attributes  to  a real-world  domain,  such  as  electronics  equipment  trouble- 
shooting. 
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TR  No.  83 


Nicholas  A.  Bond,  Jr.,  William  T.  Gabrielli,  8.  Joseph  W.  Rigney. 

Studies  of  verbal  problem-solving:  I.  Two  performance-aiding 

programs,  August  1977. 

Two  computer  programs  were  written  to  provide  on-line  aiding  to 
human  problem  solvers.  Both  programs  were  written  in  time-shared  BASIC, 
and  were  designed  for  "membership"  problems.  In  this  kind  of  problem, 
there  are  several  English  sentences  and  implicit  in  the  sentences  are 
various  relations;  the  task  is  to  infer  a membershiD  structure  that  is 
compatible  with  all  the  logical  constraints.  Membership  problems  may 
be  cast  in  various  settings,  such  as  a murder  mystery  where  a culprit  is 
to  be  identified. 

One  program  (FIRST)  was  based  on  Findler's  "Universal  Puzzle 
Solver"  concept;  the  other  (GABE)  used  Wang's  theorem-orover  loqic. 

In  both  programs,  the  human  operator  converted  English  problem  sentences 
to  logical  membership  relations.  The  programs  kept  track  of  all  rela- 
tions entered,  indicated  when  more  data  inputs  were  needed,  and  scored 
whether  a correct  answer  was  achieved. 

Of  the  two  programs,  FIRST  appears  to  be  most  feasible  with 
ordinary  college  subjects.  It  accepts  logical  inputs  in  a near-English 
format,  and  shows  current  logical  status  of  a problem  via  tabular  arrays 
of  X's  and  0's.  The  present  version  of  GABE  used  a strict  "p,  q,  r" 
logical  notation;  college  subjects  find  this  difficult  and  unsatis- 
factory . 

The  structure  of  the  FIRST  program  suggests  a "denth-of-inference" 
measurement  technique.  When  all  possible  logical  paths  in  a membershio 
problem  are  known,  the  "depth"  of  any  given  node  in  the  path  can  be 
obtained  from  probabi 1 i ty-of-success  numbers  at  that  node;  also  it 
appears  that  a subject's  logical  progress  along  a path  can  be  commuted 
and  displayed.  Further  empirical  work  will  explore  the  usefulness  of 
such  depth  measures  for  scoring  individual  performances , and  for 
teaching  problem-solving  heuristics  in  technical  materials. 


TR  No.  87 


Nicholas  A.  Bond,  Donald  McGregor,  Kathy  Schmidt,  Mary  Lattimore,  & 

Joseph  W.  Rigney,  Studies  of  verbal  problem-solving:  II.  Predic- 
tion of  performance  from  sentence-processing  scores,  June  1978. 

In  complex  reasoning  problems  of  "the  who-done-it"  type,  four 
distinct  solution  processes  were  identified: 
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(1)  intra-sentence  or  word- into- symbol  processing,  where  the 
solver  converts  the  verbal  information  into  strict  logical 
relations ; 

(2)  inter-sentence  processing,  where  the  subject  has  to  combine 
the  logic  from  two  or  more  sentences  in  order  to  obtain  new 
i nferences ; 

(3)  orderi ng  of  problem  variables  into  some  rank  or  numerical 
ordering  scheme; 

(4)  collect!’ ng  the  logical  relations  into  a reliable  format 
that  will  reduce  the  memory  load  and  facilitate  the 
"where-to-look-next"  decision. 

This  study  explored  the  extent  to  which  separate  scores  on  these  processes 
could  predict  performance  on  difficult  problems. 

Scores  on  the  sentence-logic  items  correlated  well  (r=.68,  N=34) 
with  number  of  reasoning  problems  solved,  as  did  the  ordering  score 
(r=. 75).  These  scores,  then,  presumably  are  "closer"  to  the  actual 
performance  than  are  verbal  scores  such  as  McGraw-Hill  Reading  Rates 
(r=.40  to  .50).  Individual  timing  of  inference  responses  showed  that 
subjects  often  had  long  pauses  during  inter-sentence  processina,  whereas 
intra-sentence  responding  was  relatively  fast  and  regular.  The  inter- 
sentence portions  of  the  performance  appeared  to  be  key  discriminators 
between  success  and  failure. 

A small  training  experiment  was  carried  out  with  seven  new  subjects, 
who  were  matched  on  reading  scores  with  the  previous  group.  These  sub- 
jects were  given  six  hours  of  intensive,  individual  practice  on  the  four 
processes;  a standard  matrix  format  was  used,  and  five  rules  and  heuristics 
were  taught  which  were  supposed  to  facilitate  inter-sentence  reasoning. 

The  trained  people  did  show  improved  sentence-logic  scores  (median  about 
40  over  the  comparison  group);  and  if  a large  reasoning  problem  contained 
only  straightforward  sentences,  then  the  training  was  very  effective. 

In  fact,  all  seven  subjects  solved  correctly  a 4-dimension,  5-variable 
negative-disjunction  problem  within  a few  minutes.  For  those  problems 
which  hinged  upon  appreciation  of  verbal  subtleties,  though,  the  training 
did  not  help  at  all. 

The  investigation  supports  the  idea  of  rapidly  teaching  some 
"logical  tricks"  in  higher-order  cognitive  operations;  but  the  special 
training  only  works  if  the  problem  material  is  clean  and  unambiguous. 

One  obvious  extension  of  the  study  is  to  see  if  the  same  increase  in 
performance  can  be  produced  in  a practical-reasoning  domain  such  as 
troubleshooting  of  digital  devices;  another  extension  is  to  look  more 
closely  at  the  verbal  subtleties  which  so  effectively  prevent  solution 
of  some  large  problem. 
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VII.  MAINTENANCE  TRAINER-SIMULATOR  DEVELOPMENT 


Work  in  this  area  proceded  throughout  the  term  of  the  contract, 
building  upon  previous  research  on  computer-control  1 ed  trainer  simulators 
done  at  BTL  under  ONR/ARPA  funding.  Two  technical  reports  were  produced. 
These  reports  describe  a revised  trainer-simulator  called  the  Generalized 
Maintenance  Trainer-Simulator  (GMTS),  and  report  on  two  field  evaluations 
of  the  system.  The  present  contract  supported  the  development  of  the 
GMTS  system  and  programming.  Development  of  data  bases  and  the  conduct 
of  the  field  tests  were  jointly  sponsored  by  ONR  and  by  the  Naval 
Personnel  Research  and  Development  Center. 

The  philosophy  of  the  GMTS  system  is  that  there  should  be  a rela- 
tively low-cost,  stand-alone  system  for  intensive  practice  in  trouble- 
shooting. Furthermore,  the  hardware  and  software  should  be  general- 
purpose.  That  is,  it  should  be  possible  to  enter  a data  base  for  a new 
piece  of  equipment  and  provide  students  with  practice  on  the  new  equip- 
ment without  making  changes  to  the  hardware  delivery  system  or  the 
GMTS  p"-ogra^s.  The  two  technical  reports  produced,  taken  together, 
constitute  evidence  that  GMTS  has  this  general  nature.  The  system  was 
used  first  to  provide  practice  in  systems  level  troubleshooting  in  the 
UHF  communications  side  of  the  Fleet  Communications  System.  The  system 
was  tested  with  this  data  base  on  twenty  students  about  to  enter  class  C 
schools.  The  second  evaluation  of  the  system  was  performed  with  the 
AN/SPA-66  Radar  Repeater;  ten  subjects  used  this  system  for  trouble- 
shooting practice,  then  worked  actual  troubleshooting  problems  inserted 
into  an  AN/SPA-66  equipment.  In  both  cases  the  same  hardware  and  GMTS 
program  was  used,  but  with  different  data  bases. 


Development  work  is  now  procedi ng,  under  the  sponsorship  of  NPRDC 


and  ARF’A,  to  convert  the  GMTS  system  to  current  generation  hardware, 
to  develop  a new  data  base,  and  to  implement  and  test  the  new  system 
in  a Class  C school  environment. 


TR  No.  89 

Joseph  W.  Rigney,  Douglas  M.  Towne,  Carole  A.  King,  & Patrick  J.  Moran, 
Field  evaluation  of  the  generalized  maintenance  trainer-simulator : 
I.  Fleet  communications  system,  August  1978. 


The  Generalized  Maintenance  Trainer-Simulator  (GMTS)  is  a concept 
for  giving  students  in  Class  C schools  intensive  practice  in  trouble- 
shooting equipment  and  systems  taught  in  those  schools.  It  can  be  used 
for  any  device  in  which  signal  paths  and  their  relationships  to  controls, 
indicators,  and  test  points  can  be  defined.  The  GMTS  uses  generative 
CAI.  That  is,  it  generates  the  interaction  with  the  student  by  re- 
ferring to  his  last  inputs  and  to  its  stored  history  of  interactions 
with  him  up  to  that  point.  To  the  extent  that  these  individual  student 
histories  constitute  models  of  individual  students,  GMTS  constructs  a 
model  of  each  student  to  interact  with  that  student.  In  addition  to 
these  features,  GMTS  is  uniquely  suitable  for  use  in  Class  C school 
training.  The  computer  program  that  implements  the  instructional 
system  is  indifferent  to  the  specific  equipment  being  taught.  What 
specific  equipment  that  is  simulated  by  the  GMTS  is  determined  by 
loading  two  data  bases  for  that  equipment:  one  describing  essential 
internal  features  of  the  equipment  and  the  other  describing  the  external 
features . 

This  is  a report  of  a field  evaluation  of  the  GMTS  applied  to 
systems  level  troubleshooting  in  the  UHF  communications  side  of  the 
Fleet  Communications  System.  Twenty  Class  A school  students  waiting 
to  enter  C schools  practiced  solving  thirty-five  troubleshooting  problems. 
Results  were  generally  positive.  The  students  became  uniformly  more 
fluent  at  troubleshooting;  mean  times  to  solve  a problem  were  decreased 
by  a factor  of  two,  and  standard  deviations  of  these  times  were  decreased 
by  a factor  of  five.  Also,  students'  attitudes  toward  the  trainer  were 
generally  favorable.  An  additional  field  trial  is  underway,  using  the 
AN/SPA-66  radar  repeater  as  the  subject  matter. 
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TR  No.  90 


Joseph  W.  Rigney,  Douglas  M.  Towne,  Patrick  J.  Moran,  & Richard  A. 
Mishler,  Field  evaluation  of  the  generalized  maintenance 
j trainer-simulator:  II.  AN/SPA-66  radar  repeater,  November,  1978. 


This  is  a report  of  a field  evaluation  of  the  Generalized  Maintenace 
Trainer-Simulator  (GMTS)  applied  to  troubleshooting  the  AN/SPA-66  Radar 
Repea  ter. 

The  GMTS  is  a relatively  low  cost,  stand-alone  system  for  providing 
intensive  practice  in  troubleshooting.  While  GMTS  is  both  hardware  and 
computer  software,  the  computer  programs  and  associated  data  base  represent 
the  essence  of  the  GMTS  concept.  This  concept  can  be  applied  to  any 
equipment  or  system  by  compiling  the  particular  effects  of  indicators 
in  various  configurations  and  modes,  and  by  preparing  microfiche  images 
of  the  equipment  in  a multitude  of  normal  and  abnormal  states.  This 
information,  contained  in  a data  base  dedicated  to  the  equioment  to  be 
simulated,  provides  the  GMTS  program  everything  necessary  to  generate 
and  present  meaningful  interactions  with  each  individual  student.  Since 
the  program  itself  does  not  contain  data  specific  to  any  simulated 
system,  the  trainer-simulator  is  termed  'generalized.' 

This  field  test  was  the  second  to  be  conducted;  the  first  involved 
twenty  Class  A school  students  troubleshooting  the  UHF  communications 
side  of  the  Fleet  Communications  System.  A major  objective  of  the 
second  field  test  was  to  demonstrate  the  generality  of  the  trainer- 
simulator,  thus  a completely  different  equipment  was  implemented.  The 
data  base  for  this  target  equipment  was  constructed  entirely  by  two 
technicians  who  received  brief  training  in  GMTS  data  base  formulation 
(whereas  the  data  base  for  the  first  field  tesr  was  assembled  primarily 
by  the  personnel  responsible  for  programming  GMTS). 

The  field  test  involved  ten  subjects,  each  attempting  to  isolate 
thirty-three  simulated  malfunctions  over  a sixteen-hour  period.  Following 
this  practice  phase  the  students  were  tested  using  an  actual  AN/SPA-66 
with  actual,  inserted,  malfunctions. 

As  with  the  first  field  test,  results  are  generally  positive, 
especially  in  relation  to  success  in  the  test  phase  using  actual  equip- 
ment. Owing  to  the  small  sample  size,  however,  this  field  test  is  more 
valuable  in  assessing  the  success  with  which  GMTS  can  implement  a wide 
range  of  target  systems. 
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VIII.  RECOMMENDATIONS 

| 

All  of  the  areas  of  research  pursued  under  this  contract  have 
the  potential  for  producing  products  useful  to  the  Navy.  Specific 
recommendations  can  be  made  with  respect  to  five  of  these  research 
areas,  the  products  of  which  differ  in  their  current  applicability. 

One  of  the  research  efforts,  the  development  of  the  maintenance  trainer- 
simulator,  has  yielded  a product  which  is  essentially  ready  for  imple- 
mentation in  variety  of  Navy  school  environments.  The  discussions  of 
the  five  research  areas  for  which  recommendations  can  be  made  are 
arranged  roughly  in  order  of  the  immediate  applicability  of  their 
results,  with  the  most  immediately  applicable  research  products  dis- 
cussed last. 

Instructional  Sequence  Optimization 

The  research  discussed  in  Section  III,  above,  resulted  in  a 
system  for  optimizing  an  instructional  sequence  with  respect  to  time 

y ' 

in  a CAI  context.  While  this  method  did  not  presume  that  time  was  the 
only  crucial  resource  for  optimization,  the  work  demonstrated  the 
difficulty  of  experimentally  exercizing  an  optimization  technique  in 

Y 

which  only  a sub-set  of  acknowledged  key  variables  are  involved.  Other 
factors  which  should  be  included  in  the  objective  function  include  the 
cost  of  an  instructional  sequence,  the  clarity  of  targeted  concepts  in 
the  instructional  domain,  and  performance  measures  of  the  student's 
grasp  of  the  instructional  content. 

A J 
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It  is  to  be  expected  that  the  armed  services,  as  large  organiza- 
tions responsible  for  vast  amounts  of  technical  instruction,  will  make 
increasing  use  of  CAI  systems  in  the  future.  Features  of  CAI  which  will 
motivate  such  greater  utilization  will  include  low  cost,  standardized 
instruction,  and  the  opportunity  for  time-savinq  but  still  inexpensive 
individualized  instruction.  As  the  extent  of  CAI  grows  in  the  armed 
services  it  will  become  ever  more  important  to  be  able  to  manage  instruc- 
tion in  automatic  and  cost-effective  ways.  One  aspect  of  such  effective 
management  will  be  instructional  sequence  optimization.  Although  the 
results  of  the  research  carried  out  to  date  do  not  justify  an  immediate 
development  program  to  produce  a comouter-managed  instructional  system 
using  instructional  sequence  optimization  techniques  for  the  Navy,  the 
potential  benefits  to  be  derived  from  eventually  implementing  such  a 
system  are  great.  Support  of  further  research  in  the  area,  particularly 
research  emphasizing  several  features  of  the  instructional  sequence  to 
be  optimized,  is  indicated. 

He u ristic  Techniques  for  Problem  Solving 

BTL  research  in  this  domain,  described  in  Section  IV  above,  has 
shown  that  logical  problems  couched  in  verbal  form  may  present  peculiar 
obstacles  for  solution  because  of  that  form.  This  is  an  issue  of  con- 
cern for  the  military,  because  many  technical  problems  and  prepared 
information  sources  for  solving  technical  problems  are  presented  in 
a verbal  format.  A technician  who  must  operate  or  troubleshoot  an 
electronic  device,  for  example,  is  likely  to  talk  and  think  about  his 
actions  in  qualitative  verbal  terms.  His  attempts  to  validate  his 
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interpretations  of  the  results  of  his  tests  of  the  equipment  may  be 
confirmed,  guided,  contradi cted,  or  frustrated  by  the  information  in 
a verbal  format  in  technical  manuals.  Our  results  indicate  that  with 
a "clean"  problem  (that  is,  one  that  is  well  defined  and  expressed  ir. 
simple  declarative  sentences)  almost  any  regular  aidinq  format  will 
assure  rapid  and  accurate  solution.  This  result  leads  to  the  practical 
possibility  of  requiring  technical  materials  to  be  written  in  such  a 
way  that  the  text  is  "clean." 

In  the  complex  verbal  puzzle  problem  domain,  a key  skill  is  the 
identification  of  phrases  which  essentially  split  the  problem  into 
smaller  and  more  manageable  parts.  For  instance,  if  the  problem  has 
six  variables  which  have  to  be  sorted  or  identified  in  some  way,  there 
are  often  one  or  two  key  sentences  which  permit  simple  categorization 
of  these  variables  into  subclasses.  The  skills  required  to  identify  and 
to  exploit  these  high-information  sentences  have  not  been  studied,  but 
there  is  reason  to  expect  that  such  skills  are  highly  developed  in  the 
truly  expert  problem  solver.  Perhaps  the  skill  is  analogous  to  that 
shown  by  the  troubleshooter  who  makes  a few  rapid  narrowing-down  checks 
to  attain  crude  isolation  of  a malfunction.  Once  identified  and  under- 
stood, such  skills  should  be  explicitly  taught  to  students  such  as 
electronics  equipment  technicians,  who  must  learn  to  be  effective 
problem  sol  vers . 

A second  major  aspect  of  the  research  findings  in  this  area  has 
to  do  with  the  use  of  automatic  problem-solving  aids.  Complex  intel- 
lectual tasks,  such  as  combining  probability  information,  controlling 
several  aircraft,  or  troubleshooting  electronic  equipment  could  be  made 
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easier  for  those  assigned  these  tasks  through  the  use  of  appropriate 
computer-based  aid  systems.  Such  systems  would  perform  many  library, 
calculating,  and  bookkeeping  chores;  the  controlling  humans  would, 
of  course,  make  the  decisions  and  control  the  course  of  processing. 

Our  research  in  this  area  can  be  thought  of  as  a first  step  toward 
providing  automatic  systems  to  aid  in  logical  problem  solution.  What 
is  required  now  is  an  effort  to  develop  such  aiding  devices  for  prac- 
tical military  tasks  and  to  test  their  effectiveness. 

Visual  Analogies 

The  research  reported  in  Section  II  yielded  results  which  suggest 
applications  to  the  development  of  CAI  technical  subject  matter  course- 
ware. CAI  permits  the  use  of  interactive  graphics  diSDlays  to  help 
convey  the  nature  of  complex  processes  to  students.  The  results  of 
our  research  indicate  that  the  use  of  such  displays  is  most  effective 
for  initial  presentation  of  the  concepts.  Interactive  graphics  is 
less  useful  as  a review  device  for  student  use.  The  research  has  shown 
that  a more  effective  post-presentation  learning  activity  is  guided 
student  reconstruction  of  a graphic  display.  This  can  be  accomplished 
through  the  use  of  a touch  panel  or  a sonic  pen  device,  either  of  which 
takes  advantage  of  the  students'  natural  pointing  responses.  Further 
research  is  now  needed  to  determine  what  kinds  of  complex  tODics 
benefit  most  from  interactive  graphics  presentation  and  from  student 
reconstruction  of  graphics;  Such  findings  could  then  be  applied  to  the 
teaching  of  Navy  technical  subject  matter. 

The  low  costs  and  other  attractive  features  of  CAI  are  likely  to 
ensure  increased  use  of  this  instructional  technique  and  medium  in  the 
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armed  services.  These  features,  however,  will  not  necessarily  ensure 
that  the  most  effective  use  of  the  capabilities  of  the  instructional 
computer  system  will  be  used.  It  is  possible  that,  in  some  cases, 
attempts  will  be  made  to  simply  transport  a traditional  training  manual 
approach  to  the  new  CAI  courses.  Such  a course  would  fail  to  make  use 
of  such  features  as  interactive  graphics.  Our  results  have  shown  that 
this  feature  can  improve  learning  over  a more  traditional  approach. 

Further  research  could  be  expected  to  lead  to  the  production  of  guide- 
lines for  the  effective  use  of  interactive  graphics  and  guided  student 
constructed  graphics.  Implementation  of  such  guidelines  would  then  lead 
to  more  effective  CAI  courseware. 

Cognitive  Strategies  for  Text  Processing 

Toward  the  end  of  the  course  of  research  reported  in  Section  V 
an  unanticipated  result  was  discovered  that  may  have  important  applica- 
tions to  the  use  of  texts  for  instructional  purposes.  The  results  of 
three  experiments  indicated  that  understanding  of  and  memory  for  the 
contents  of  different  types  of  texts  benefit  differentially  from  par- 
ticular text-processi ng  techniques.  For  example,  memory  for  texts 
that  have  an  explanatory  nature  benefits  from  the  processes  of  re- 
reading and  summarizing  texts.  Memory  for  instructional  texts  is  not 
significantly  improved  through  such  techniques.  These  findings  must  be 
thought  of  as  tentative,  but  if  they  can  be  replicated  in  experiments 
of  the  appropriate  design,  their  implications  will  have  obvious  applica- 
tions for  military  instruction.  Either  of  two  methods  could  be  expected 
to  prove  the  more  effective  means  of  using  the  results  to  improve  learn- 
ing from  instructional  texts.  Texts  could  be  prepared  which  incorporate 
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directions  to  the  student  as  to  which  techniques  should  be  applied  to 
each  segment  of  text  in  order  to  maximize  learning  from  that  segment. 
Alternatively,  students  could  be  trained,  in  a short  course,  to  recog- 
nize texts  of  each  major  type  and  to  apply  the  appropriate  techniques. 
Further  research  is  required  before  either  of  these  methods  could  be 
developed  for  application  in  Navy  schools.  First,  more  must  be  learned 
about  the  relationships  between  text  type  and  effective  text  processing 
techniques.  Second,  experiments  must  be  performed  to  learn  whether  the 
improvement  in  learning  from  Navy  texts  brought  about  through  the  use 
of  these  techniques  is  sufficiently  great  to  justify  the  development 
of  an  application.  Third,  it  must  be  determined  how  these  findings 
may  best  be  applied.  The  application  of  the  findings  of  such  a research 
program  can  be  expected  to  measurably  improve  the  efficiency  with  which 
military  students  learn  from  technical  documents. 

The  Generalized  Maintenance  Trainer-Simulator 

The  generalized  maintenance  trainer-s’mulator  (GMTS)  is  the  end- 
product  of  many  years  of  research  funded  by  the  Office  of  Naval  Research 
and  the  Defense  Advanced  Research  Projects  Agency.  The  research  con- 
ducted on  this  contract  has  shown  that  GMTS  is  an  effective  way  of  in- 
creasing the  amount  of  troubleshooting  practice  on  particular  equipments 
for  Navy  electronics  students.  It  also  provides  the  capability  of 
simulating  a wider  range  of  troubleshooting  experiences  that  can  be 
provided  practically  on  real  equipment.  The  research  has  shown  that 
data  bases  for  new  types  of  electronic  equipment  can  be  prepared  for 
the  GMTS  system  by  subject-matter  experts  who  are  not  computer  programmers. 
Two  field  evaluations  in  Navy  schools  have  shown  that  use  of  GMTS  results 
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in  more  efficient  use  of  students 's  time  and  in  time  savings  for  Navy 

* 

instructors,  while  providing  students  with  extensive  practice  at  low 
cost. 

The  GMTS  system  is  essentially  ready  for  general  use  in  Navy 
Class  C schools.  Unfortunately,  however,  in  its  current  implementation 
it  makes  use  of  outmoded  and  unreliable  computer  equipment.  Behavioral 
Technology  Laboratories  began  research  on  a new  contract,  jointly 
funded  by  ‘.he  Naval  Personnel  Research  and  Development  Center  and  the 
Defense  Advanced  Research  Projects  Agency,  on  September  30,  1973 
(N001 23-78-C- 1 457 ) to  (1)  develop  the  GMTS  software  on  current-generati on 
hardware  (Terak  8510/A  microcomputer),  (2)  prepare  new  data  bases  for 
the  new  GMTS  system,  and  (3)  to  implement  and  test  the  effectiveness  of 
the  new  system  in  a Class  C school.  The  development  of  the  new  software 
will  have  special  consequences  for  the  future  and  onqoing  usefulness  of 
the  GMTS  system  to  the  Navy.  The  new  software  will  be  transportable 
to  other  computers;  therefore,  the  GMTS  system  will  be  protecte-i  against 
hardware  obsolescence.  This  will  be  accomplished  by  devel op’ug  the  new 
GMTS  software  on  the  UCSD  Pascal  operating  system,  a virtually  machine- 
independent  operating  system  for  micro-  and  mini-computers. 

We  recommend  that,  once  the  initial  development  phase  of  the  new 
GMTS  system  is  complete  (approximately  October  1979),  data  base  develop- 
ment for  a variety  of  Navy  Electronic  equipments  be  developed  for  wide- 
spread use  in  Class  C schools.  We  further  recommend  that  research  on 
the  potential  usefulness  of  the  GMTS  system  in  Class  A schools  be 
conducted.  The  GMTS  system  can  be  expected  to  serve  as  an  important 
component  of  all  Navy  electronics  equipment  maintenance  training. 
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1 Office  of  Naval  Research 
Code  200 

Arlington,  VA  22217 

1 Office  of  Naval  Research 
Code  437 

800  N.  Quincy  St. 

Arlington,  VA  22217 

1 Scientific  Director 

Office  of  Naval  Research 
Scientific  Liaison  Group/Tokyo 
American  Embassy 
AFO  San  Francisco,  CA  96503 

1 Dr.  Kneale  Marshall 

Scientific  Advisor  to  the  Chief 
of  Naval  Personnel 
Naval  Bureau  of  Personnel  (Pers  OR) 
Rm.  4410,  Arlington  Annex 
Washington,  DC  20370 

1 Dr.  Richard  A.  Pollack 
Academic  Computing  Center 
U.S.  Naval  Academy 
Annapolis,  MD  21 402 

1 Mr.  Arnold  I.  Rubinstein 

Human  Resources  Program  Manager 
Naval  Material  Command  (034^) 

Room  1044,  Crystal  Plaza  *5 
Washington,  DC  20360 

1 Dr.  Worth  Scanland 

Chief  of  Naval  Education  & Training 
Code  N-5 

NAS,  Pensacola,  FL  32508 
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Navy  (Cont'd) 

1 A.  A.  Sjoholm 

Tech.  Support,  Code  201 
Navy  Personnel  R&D  Center 
San  Diego,  CA  92152 

1 Mr.  Robert  Smith 

Office  of  Chief  of  Naval 
Operations 
OP- 987  E 

Washintton,  DC  20350 

Dr.  Alfred  F.  Smode 

Training  Analysis  & Evaluation 
Group  (TAEG) 

Dept,  of  the  Navy 
Orlando,  FL  32813 

1 CDR  Charles  J.  Theisen,  Jr. 

MSC , USN 

Head,  Human  Factors  Engineering 
Laboratory 

Naval  Air  Development  Center 
Warminster,  PA  18974 


Dr.  Ralph  Dusek 
U.S.  Army  Research  Institute 
5001  Eisenhower  Ave. 
Alexandria,  VA  22333 

Dr.  Frank  J.  Harris 
U.S.  Army  Research  Institute 
5001  Eisenhower  Ave. 
Alexandria,  VA  22333 

Dr.  Milton  S.  Katz 
Individual  Training  & Skill 
Evaluation  Technical  Area 
U.S.  Army  Research  Institute 
5001  Eisenhower  Ave. 
Alexandria,  VA  22333 

Dr.  Harold  F.  O'Neil,  Jr. 
Attn:  PERI -OK 
5001  Eisenhower  Ave. 
Alexandria,  VA  22333 


Di rector 

Training  Development 

U.S.  Army  Administration  Center 

Attn:  Dr.  Sherrill 

Ft.  Benjamin  Harrison,  IN  46218 

Dr.  Joseph  Ward 

U.S.  Army  Research  Institute 

5001  Eisenhower  Ave. 

Alexandria,  VZ  22333 


Air  Force 

1 Air  University  Library 
AUL/LSE  76/443 
Maxwell  AFB , AL  36112 

1 Dr.  T.  E.  Cotterman 
AFHRL/ASR 

Wright  Patterson  AFB 
Ohio  -5433 

1 Dr.  G.  A.  Eckstrand 
AFHRL/AS 

Wright-Patterson  AFB 
Ohio  45433 

1 CDR.  Mercer 

CNET  Liaison  Officer 
AFHRL/Flying  Training  Div. 
Williams  AFB,  AZ  85224 

1 Dr.  Ross  L.  Morgan  (AFHRL/ASR) 
Wright-Patterson  AFB 
Ohio  45433 

1 Dr.  Marty  Rockway  (AFHRL/TT) 
Lowry  AFB 
Colorado  80230 

1 Jack  A.  Thorpe,  Capt.  USAF 
Program  Manager 
Life  Sciences  Directorate 
AFOSR 

Bolling  AFB,  DC  20332 

1 Brian  K.  Waters,  LCOL,  USAF 
Air  University 
Maxwell  AFB 
Montgomery,  AL  36112 
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Marines 
1 Director 

Office  of  Manpower  Utilization 
HQ,  Marine  Corps  (MPU) 

BCB,  Bldg.  2009 
Quantico,  VA  22134 

1 Dr.  A.  L.  Slafkosky 

Scientific  Advisr  (Code  RD-1 ) 
HQ,  U.S.  Marine  Corps 
Washington,  DC  20380 


Coast  Guard 

1 Mr.  Joseph  J.  Cowan,  Chief 

Psychological  Research  (G-P-l/62) 
U.S.  Coast  Guard  HQ 
Washington,  DC  20590 


Other  DoD 

1 Dr.  Stephen  Andriole 

Advanced  Research  Projects  Agency 
1400  Wi Ison  Blvd. 

Arlington,  VA  22209 

12  Defense  Documentation  Center 
Cameron  Station,  Bldg.  5 
Alexandria,  VA  22314 
Attn:  TC 

1 Dr.  Dexter  Fletcher 

Advanced  Research  Projects  Agency 
1400  Wilson  Blvd. 

Arlington,  VA  22209 

1 Military  Assistant  for  Human 
Engineering 

Office  of  the  Director  of  Defense 
Research  & Engineering 
Room  3D129,  The  Pentagon 
Washington,  DC  20301 

1 Director,  Research  & Data 
OSD/MRA&L  (Rm.  3B919) 

The  Pentagon 
Washington,  DC  20301 


Civil  Government 

1 Dr.  Joseph  Markowitz 

Office  of  Research  & Development 
Central  Intelligence  Agency 
Washington,  DC  20205 

1 Dr.  Andrew  R.  Molner 

Science  Education  Dev.  and 
Research 

National  Science  Foundation 
Washington,  DC  20550 

1 Dr.  Wallace  Sinaiko,  Director 

Manpower  Research  & Advisory  Service 
Smithsonian  Institute 
801  N.  Pitt  Street 
Alexandria,  VA  22314 

1 Dr.  Joseph  L.  Young,  Director 
Memory  & Gognitive  Processes 
National  Science  Foundation 
Washington,  DC  20550 


Non-Government 

1 Prof.  Earl  A.  Alluisi 
Dept,  of  Psychology 
Code  287 

Old  Dominion  University 
Norfolk,  VA  23508 

1 Dr.  Michael  Atwood 

Science  Applications  Institute 
40  Denver  Tech.  Center  West 
7935  E.  Prentice  Ave. 

Englewood,  CO  80110 

1 Psychological  Research  Unit 
Dept,  of  Defense  (Army  Office) 
Campbell  Park  Offices 
Canberra,  ACT  2600 
Austral ia 

1 Ms . Carole  A.  Bagley 

Minnesota  Educational  Computinq 
Consorti urn 
2520  Broadway  Dr. 

St.  Paul,  MN  55113 
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Non-Government  ( Con t ' d . ) 1 

1 Mr.  Samuel  Ball 

Educational  Testing  Service 
Princeton,  NJ  08540  1 

1 Dr.  Nicholas  A.  Bond,  Jr. 

Dept,  of  Psychology 
Sacramento  State  College 
6000  Jay  St. 

Sacramento,  CA  95819  1 

1 Dr.  John  Brackett 
SofTech 

460  Totten  Pond  Road 

Waltham,  MA  02154  1 

1 Dr.  Robert  K.  Branson 
1A  Tul ly  Bui  1 d i ng 
Florida  State  University 
Tallahassee,  FL  32306  1 

1 Dr.  John  Seeley  Brown 

Bolt  Beranek  8 Newman,  Inc. 

50  Moulton  St. 

Cambridge,  MA  02138  1 

1 Dr.  C.  Victor  Dunderson 
WICAT,  Inc. 

University  Plaza,  Suite  10 
1160  So.  State  St. 

Orem,  UT  84057  1 

1 Charles  Myers  Library 
Linvingston  House 
Linvingstone  Road 
Stratford 

London  E15  2LJ  1 

England 

1 Dr.  Kenneth  E.  Clark 

College  of  Arts  8 Sciences 
University  of  Rochester 
River  Campus  Station  1 

Rochester,  NY  14627 

1 Dr.  Allan  M.  Collins 

Bolt  Beranek  8 'Jew man,  Inc. 

50  Mpulton  St.  1 

Cambridge,  MA  02138 


ERIC  Facility-Acquisitions 
4833  Rugby  Ave. 

Bethesda,  MA  20014 

Major  I.  N.  Evonic 

Canadian  Forces  Pers.  Apnlied  Research 
1107  Avenue  Road 
Toronto,  Ontario 
Canada 

Dr.  Victor  Fields 
DeDt.  of  Psychology 
Montgomery  College 
Rockville,  MD  20850 

Dr.  Edwin  A.  Fleishman 
Advanced  Research  Resources  Organ. 

8555  Sixteenth  St. 

Silver  Spring,  MD  20910 

Dr.  John  R.  Frederiksen 
Bolt  Beranek  8 Newman,  Inc. 

50  Moulton  St. 

Cambridge,  MA  02138 

Dr.  Frederick  C.  Frick 
MIT  Lincoln  Laboratory 
Room  D 268 
P.0.  Box  73 
Lexington,  MA  02173 

Dr.  Vernon  S.  Gerlach 
College  of  Education 
146  Payne  Bldg.  B 
Arizona  State  University 
Tempe,  AZ  35281 

Dr.  Robert  Glaser 
LRDC 

University  of  Pittsburgh 
3939  O'Hara  St. 

Pittsburgh,  PA  15213 

Dr.  Ira  Goldstein 

Xerox  Palo  Alto  Research  Center 

3333  Coyote  Road 

Palo  Alto,  CA  94304 

Dr.  James  G.  Greeno 
LRDC 

University  of  Pi ttsburah 
3939  O'Hara  St. 

Pittsburgh,  PA  15213 
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Non-Government  ( Co n t ' d . ) 

1 Dr.  Richard  S.  Hatch 
Decision  Systems  Asso.,  Inc. 

350  Fortune  Terrace 
Rockville,  MD  20854 

2 Dr.  Barbara  Hayes-Roth 
The  Rand  Corporation 
1700  Main  St. 

Santa  Monica,  CA  90406 

1 Library 

HumRRO/Western  Division 
27857  Berwick  Dr. 

Carmel,  CA  93921 

1 Dr.  Earl  Hunt 

Dept,  of  Psychology 
University  of  Washington 
Seattle,  WA  98105 

1 Dr.  Lawrence  B.  Jonnson 

Lawrence  Johnson  & Asso.,  Inc. 
Suite  502 
2001  S Street  NW 
Washington,  DC  20009 

1 Dr.  Wilson  A.  Judd 

McDonnell-Douglas  Astronautics 
Company  East 
Lowry  AFB 
Denver,  CO  80230 

1 Dr.  Arnold  F.  Kanarick 
Honeywell,  Inc. 

2600  Ridgeway  Pkwy 
MN  55413 

1 Dr.  Roger  A.  Kaufman 
203  Dodd  Hall 
Florida  State  University 
Tallahassee,  FL  32306 

1 Mr.  Marlin  Kroger 
1117  Via  Goleta 

Palos  Verdes  Estates,  CA  90274 

1 LC0L  C.R.J.  LaFleur 

Personnel  Applied  Research 
National  Defense  HQS 
101  Colonel  by  Drive 
Otawa,  Canada  K1A  0K2 


1 Dr.  Robert  A.  Levit 
Manager, 

Behavioral  Sciences 
The  BDM  Corporation 
7915  Jones  Branch  Dr. 

McClean,  VA  22101 

1 Dr.  Robert  R.  Mackie 

Human  Factors  Research,  Inc. 

6780  Cortona  Dr. 

Santa  Barbara  Research  Park 
Goleta,  CA  93017 

1 Mr.  Mark  Miller 

Massachusetts  Institute  of  Technology 
Artifial  Intelligence  Lab 
545  Tech  Square 
Cambridge,  MA  02139 

1 Dr.  Richard  B.  Millward 
Dept,  of  Psychology 
Hunter  Lab. 

Brown  University 
Providence,  RI  82912 

1 Dr.  Stuart  Milner 

Department  of  Education 
George  Mason  University 
4400  Fairfax  Drive 
Fairfax,  VA  22030 

1 Dr.  Allen  Munro 

Univ.  of  Southern  California 
Behavioral  Technology  Labs 
3717  So.  Hope  St. 

Los  Angeles,  CA  90007 

1 Dr.  Donald  A.  Norman 

Dept,  of  Psychology  - C-009 
Univ.  of  California,  San  Dieqo 
La  Jolla,  CA  92093 

1 Dr.  Jesse  Orlansky 

Institute  for  Defense  Analysis 
400  Army  Navy  Drive 
Arlington,  VA  22202 

1 Dr.  Seymour  A.  Papert 

Massachusetts  Institute  of  Technology 
Artificial  Intelligence  Lab 
545  Technology  Square 
Cambridge,  MA  02139 
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Non -Government  (Cont'd.) 

1 Dr.  James  A.  Paulson 

Portland  State  University 
P.0.  Box  751 
Portland,  OR  97207 

1 Mr.  A.  J.  Pesch,  President 
Eclectech  Associates,  Inc. 

P.0.  Box  178 
N.  Stonington,  CT  06359 

1 Mr.  Luigi  Petrullo 
2431  N.  Edgewood  St. 

Arlington,  VA  22207 

1 Dr.  Peter  Poison 
Dept,  of  Psychology 
University  of  Colorado 
Boulder,  CO  80302 

1 Dr.  Diane  M.  Ramsey- Klee 

R-K  Research  & Systems  Design 
3947  Ridgemont  Dr. 

Malibu,  CA  90265 

1 Dr.  Peter  B.  Read 

Social  Science  Research  Council 
605  Third  Ave. 

New  York,  NY  10016 

1 Dr.  Fred  Reif 
SESAME 

c/o  Physics  Department 
University  of  California 
Berkeley,  CA  94720 

1 Dr.  Andrew  M.  Rose 

1 American  Institute  for  Research 

1055  Thomas  Jefferson  St.  NW 
Washington,  DC  20007 

1 Dr.  Leonard  L.  Rosenbaum,  Chairman 
Department  of  Psychology 
| Montgomery  College 

Rockville,  MD  20850 

1 Dr.  Ernest  Z.  Rothkopf 

Bell  Laboratories 
600  Mountain  Avenue 
| Murray  Hill,  NJ  07974 
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1 Dr.  Walter  Schneider 
Department  of  Psychology 
University  of  Illinois 
Champai qn,  IL  61820 

1 Dr.  Allen  Schoenfeld 
SESAME 

c/o  Physics  Department 
University  of  California 
Berkeley,  CA  94720 

1 Dr.  Robert  J.  Seidel 

Instructional  Technology  Group 
HumRRO 

300  N.  Washington  St. 

Alexandria,  VA  22314 

1 Dr.  Robert  Singer,  Director 
Motor  Learning  Research  Lab 
Florida  State  University 
212  Montgomery  Gym 
Tallahassee,  FL  32306 

1 Dr.  Richard  Snow 
School  of  Education 
Stanford  University 
Stanford,  CA  94305 

1 Dr.  Robert  Sternberg 
Dept,  of  Psychology 
Yale  University 
Box  11 A , Yale  Station 
New  Haven,  CT  06520 

1 Dr.  Albert  Stevens 

Bolt  Beranek  & Newman,  Inc. 

50  Moulton  St. 

Cambridge,  MA  02138 

1 Mr.  William  Stobi e 

McDonnell-Douqlas  Astronautics  Co. 
P.0.  Box  30204 
Chico,  CA  95926 

1 Dr.  Persis  Sturgis 
Dept,  of  Psychology 
California  State  University 
Chico,  CA  95926 
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Non -Government  (Cont'd.) 


1 Mr.  D.  J.  Sullivan 

c/o  Canhon  Research  Group,  Inc. 

741  Lakefield  Road 
Westlake  Village,  CA  91361 

1 Dr.  Patrick  Suppes 

Institute  for  Mathematical  Studies 
in  the  Social  Sciences 
Stanford  University 
Stanford,  CA  94305 

1 Dr.  Perry  Thorndyke 
The  Rand  Corporation 
1700  Main  St. 

Santa  Monica,  CA  90406 

1 Dr.  K.  W.  Uncapher 

USC/Information  Sciences  Institute 
4676  Admiralty  Way,  #1100 
Marina  del  Rey,  CA  92091 

1 Dr.  Benton  J.  Underwood 
Dept,  of  Psychology 
Northwestern  University 
Evanston,  IL  60201 

1 Dr.  Willard  S.  Vaughan,  Jr. 
Oceanautics,  Inc. 

422  Sixth  St. 

Annapolis,  MD  21403 

1 Dr.  Robert  Vineberg 

HumRRO/Western  Division 
27857  Berwick  Dr. 

Carmel,  CA  93921 

1 Dr.  Clarie  E.  Weinstein 

Educational  Psychology  Dept. 

Univ.  of  Texas  at  Austin 
Austin,  TX  78712 

1 Dr.  David  J.  Weiss 
N660  Elliott  Hall 
University  of  Minnesota 
75  E.  River  Road 
Minneapolis,  MN  55455 
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